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ABSTRACT 

In recent years, the need to improve education has 
received considerable attention. The development of educational 
indicators is necessary to help policymakers determine whether 
educational conditions have improved and what additional actions are 
indicated. The Council of Chief State School Officers State Education 
Assessment Center initiated a project in 1986 to develop state 
indicators of the condition of science and mathematics education. 
This paper describes how this project was developed including a 
conceptual model of the educational system and state-by-state 
indicators. The rest of the paper looks at ways that indicators could 
be used in policymaking and programmatic decision making. Areas 
discutsed include: (1) "Curriculum and Instruction"; (2) "Educational 
Equity": (3) "Pre-Service Teacher Preparation"; and (4) "In-Service 
Education. " (CW) 
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INDICATORS OF SCIENCE AND MATHEMATICS EDUCATION: 
PROVIDING TOOLS FOR STATE POLICYMAKERS 

INTRODUCTION 

The need to improve K-12 education in general, and science and mathematics 
education in particular, has received considerable attention in recent years. Response to 
these concerns has been particularly evident at the state level, where many states have 
instituted changes in course offerings, requirements for high school graduation, teacher 
certification, and in-service education policies. 

Are conditions better or worse than they were before these reforms? Anecdotal 
evidence, while interesting, provides a very limited picture. As noted by the National 
Science Board Commission on Precollege Education in Mathematics, Science and 
Technology [1983], "periodic objective measurement of achievement and participation is 
essential to determine progress." The development of educational indicators is aimed at 
helping policymakers determine whether the conditions prompting the various educational 
reforms have improved, and what additional actions might be warranted. 

The Council of Chief State School Officers State Education Assessment Center 
initiated the Science/Mathematics Indicators Project in 1986 to develop state indicators of 
the condition of science and mathematics education in elementary and secondary schools. 
The project has two goals: 

(1) to improve the quality and usefulness of data on science and mathematics 
education for state policymakers and program managers 

(2) to develop a system of indicators that provides the capacity for state-by-state 
comparisons and a national database to assess the condition of science and 
mathematics education. 

During the first year of the project, information about existing science and 

mathematics education indicators was collected from each of the 50 states, the District of 

Columbia, and 3 territories (Guam, Puerto Rico, and the Virgin Islands). Using this 

information, a Task Force comprised of state science and mathematics specialists, state 
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data managers, and national experts on science and mathematics education indicators 
developed a prioritized list of indicators. High priority indicators fell into two categories: 
indicators that this project should develop with the states, and indicators that could be 
obtained through other projects such as the Department of Education's National 
Assessment of Educational Progress (NAEP) and the Schools and Staffing Survey. The 
project's year-one report includes recommendations on those indicators that should be 
developed as part of this project [Blank and Espenshade, 1987a]. 

Second year project activities have focused on working with data managers and 
science and mathemaucs specialists in each state to develop a plan for collecting data on 
thre- priority indicators. As part of a series of five Regional Meetings held in February 
and March 1988, professional staff representatives from state departments of education 
shared ideas and strategies on the use of state data at both the state and local levels. TLis 
paper elaborates on those discussions; it focuses on indicators of science and mathematics 
education and how these types of data can be used in making policy decisions at the state 
level. 

THE IMPORTANCE OF A CONCEPTUAL MODEL OF THE EDUCATION SYSTEM 
If one is attempting to describe the status of science and mathematics education, 
there is almost no limit to the amount of relevant data that could be collected. Should we 
determine the average square footage of science classrooms? The number of water 
faucets? The amount of time spent on hands-on activities? The amount of homework 
assigned? The number of windows facing south? Clearly the answers depend on one's 
view of the educational process: it is important to collect data on key aspects of the 
education system, especially on those variables that aie believed to be causally related to 
the primary outcomes of interest, e.g. student achievement We would usually not want to 
spend resources collecting data on variables that are unlikely to be related to outcomes of 
interest. 
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Oakes [1986] noted that since the pumoses of indicator systems are (1) to measure 
the health and effectiveness of the education system and (2) to help policymakers make 
better decisions, it is implicit that a set of indicators be selected 

based on an understanding of which components of the educational system art 
critical to its health and which features signal important changes in its condition. 
We should also know how the various components of the system are related to one 
another. .. Put another way, to properly specify which indicators should be parr of a 
system, we need a model of how the education system works. 

Unfortunately, while education research is making progress in understanding 
various components of the education system, the system is exceedingly complex, and there 
is not yet a single, widely accepted model of its operation. The National Research Council 
Committee on Indicators of Pretulege Science and Mathematics Education uses a simple 
model in which teacher quantity and quality and curriculum content are considered the 
major "inputs" of interest, instructional time and course enrollment the primary "processes" 
of interest, and student achievement the primary outcome of interest [Raizen and Jones, 
1985]. 

INPUTS PROCESS OLiCOME 



Teachers 
quantity 
quality 




Instructional time/ 
course enrollment 



Student 
achievement 



Curriculum 
content 



Figure 1: National Research Council 
Areas of Science and Mathematics Education to be Monitored 



Based on this model, and using a set of criteria that includes feasibility as well as 
policy-relevance of indicators, the Committee identified 8 high priority indicators: 



1. student learning in mathematics and science 

2. adult scientific and mathematical literacy 

3. time spent in elementary and junior high school science and mathematics 

4. enrollment in high school science and mathematics 
courses 

5. nature of student activities during science and mathematics instruction 

6. teacher subject matter knowledge 

7. quality of the curriculum content 

8. teacher salaries relative to those in other occupations. 

Six secondary indicators identified by the National Research Council include time 
spent on science and mathematics homework; teacher course background preparation; 
teacher time in professional activities; availability and use of materials, facilities, and 
supplies; level of federal financial support; and commitment of resources by scientific 
organizations [Murnane and Raizen, 1988J. 

The Council of Chief State School Officers (CCSSO) has developed an indicators 
model with three broad components: educational policies and practices, state 
characteristics, and educational outcomes. 

State Context or Background 
Characteristics 

Educational 

Policies c , 

Educational 

Practices Outcomes 
Figure 2: CCSSO Indicators Model 

A number of specific indicators have been identified for each of these headings. 
The category "Policies and Practices" includes measures of changeable educational 
program features: instructional time, instructional leadership, home-school relations, 
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teacher quality, per pupil expenditures, and teacher salaries. State contextual 
characteristics include size, urbanicity, and percentage of special needs students. Finally, 
attendance rates, percent graduating from high school, achievement levels, attitudes, and 
job placement are included as outcomes of interest fCCSSO, 1985]. The main feature of 
this model is that it distinguishes those inputs that are "givens" that the educational system 
must work with from those processes and policies that can be changed. Again there is 
considerable, but not total, overlap between these indicators and the ones generated using 
other models. 

The RAND Corporation has developed a comprehensive model of the education 
system that includes fiscal resources, teacher quality, and student background as inputs; 
school, curriculum, teaching, and instructional quality as processes; and participation, 
achievement, attitudes, and aspirations as outputs of interest RAND used a set of criteria 
similar to those used by the National Research Council to assess the utility of 104 factors 
that their research had identified as central to the education system; this screening process 
resulted in 40 potential indicators of the health of science and mathematics education, 
including many, but not all, of the National Research Council's priority indicators 
[Shavelson, et al., 1987]. 




Figure 3: RAND Model Linking Elements of the Educational System 
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It is clear that these three models, and a host of other existing models, have many 
similarities* - for example, they typically include components related to fiscal resources, 
teacher quality, and the curriculum but the specific components and their hypothesized 
relationships vary considerably from model to model. As Oakes [1986] notes, "More than 
any other factor, the model chosen as the basis for selecting indicators will influence what 
information an indicator system will provide." Unfortunately, we presently have too many 
models from which to choose. While there is clear research evidence supporting some 
aspects of these models (e.g., we know that increased instructional time is in fact related to 
gains in student achiev irnent), other hypothesized relationships have not r een 
demonstrated. For example, while it seems logical that teachers with a lot of subject 
matter background will do a better job than those with less pieparation, research has thus 
far been unable to document a relationship between teachers' course-taking background 
and student achievement. 

STATE-BY STATE INDICATORS: THE FIRST STEPS 

It is hard to agree on a set of state level indicators of the health of science and 
mathematics education when there are so many competing intuitive notions of how the 
system works and what components seem to make the most difference. But policy 
decisions cannot be delayed while we wait for the development of a well researched, 
consensus model of the education process; decisions are being made all of the time. While 
we are a long way from having a comprehensive, well documented causal model of the 
education process, we can nevertheless learn a lot * „ut the health of science and 
mathematics education by monitoring selected important components. 

The CCSSO Science/Mathematics Indicators Project took 3 steps to identify state- 
by-state indicators. First was development of a list of 26 "ideal" indicators in six categories: 
Student Outcomes. Instructional Time, Curriculum Content, School Conditions, Teacher 
Quality, and Resources. The list was used to design a survey with state departments of 
educatioa Each state was asked to provide information about the types of data co'lected 

ERJC « 



for each of these areas and the Instruments, data sources, and data collection procedures 
used. The results from the inventory are provided in the report "State Education Indicators 
on Science & Mathematics" [Blank and Espenshade, 1988]. 

Cost and response burden considerations make it impractical to collect state-by- 
state data on all of the "ideal" indicators. For example, teachers' knowledge of their subject 
and how to teach it has been identified as a high priority indicator, but imposing a direct 
measure such as tests for all teachers would be very difficult and a proxy measure such as 
college course preparation is available for analysis in only a few states. Priorities must be 
established and then feasibility must be assessed. For this reason, as part of the Project, 
educational researchers and state-level personnel have been involved in merging research 
findings and feasibility considerations to select a narrower set of "priority Indicators". 

The Project has chosen to start with three state-by-state indicators that are both 
important from a policy perspective, and feasible at the present time (with importance 
being judged based on our current understanding of the science/mathematics education 
process, and feasibility being judged by the willingness of a large number of states to collect 
these types of data). These are: 

o Secondary Course Enrollment in Science /Mathematics 

o Teacher Assignments in Science/Mathematics by Age, Gender, and 
Race/Ethnicity 

o Teacher Assignments by Certification Status in 
Science/Mathematics 

These choices clearly reflect a number of compromises. For example, while student 
achievement is an essential indicator of virtually any model of the education system, this 
vitally important indicator is being bypassed temporarily by the Project because state-by- 
state student outcome data is being planned as part of the National Assessment of 
Educational Progress. As another example, enrollment by gender will be reported as part 
of the course enrollment indicator for those states that collect these data; equally important 
information about enrollment by race/ethnicity will not be reported because so few states 



currently collect these data and because collecting the data presents many difficulties. 
Other state-by-state data, including indicators of teacher supply and demand, will be 
collected by the "Schools and Staffing Survey" cf the U.S. Department of Education's 
Center for Education Statistics. 

Education in the United States is primarily the responsibility of the states, which in 
turn delegate varying amounts of authority to local districts. The development of an 
indicator system is an evolutionary process, and even in the early stages the collection of 
these types of data can lead to improved policy decisions. For example, the course 
enrollment indicator will allow states to track student science and mathematics enrollment 
over time and to determine the extent to which students elect 'o take courses beyond those 
required for high school graduation Similarly, information about teacher assignments by 
certification status will all policymakers to determine the extent of out-of-field teaching 
and to monitor the severity of this problem over time. With accurate and timely data, states 
will be better able to decide whether to modify their requirements, e.g„ for high school 
graduation or teacher certification, or to use any of ? number of possible incentives as 
policy levers for improving science and mathematics education. 

STATE 2vei* T0R DATA W P ° LICY ^ PROGRAM DECISIONMAKING AT THE 

The remainder of this paper looks at some of the ways indicators of science and 
mathematics education can contribute to informed pohcymaking and programmatic 
decisions at the state level. Unless otherwise indicated, the data presetted in this section 
are from the 1985-86 National Survey of Science and Mathematics Education [Weiss, 
1987]. Information about state education policies on science and mathematics education 
and the availability of data at the state level was compiled by the CCSSO 
Science/Mathematics Indicators Project [Blank and Espenshade, 1987b, 1988]. 
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1. Curriculum and Instruction 

CCSSO data indicate that 26 states currently have state policies concerning the 
amount of instruction ir. elementary science and mathematics. For science, most states 
recommend from 20 to 30 minutes per day for grades K-3 ard from 35 to 45 minutes for 
graaes 4-6. Nationally, the average amount of time spent on science instruction is 18 
minutes per day in grades K-3 and 29 minutes per day in grades 4-6. These figures fall 
below state recommendations, and they have remained essentially the same for the 'as! 
decade, even though the need to improve science education has received a great deal of 
attention in recent years. 

Having information about time spent on elementary science available on a state-by- 
state basis would enable individual states to judge the adequacy of their results and to 
determine if the trend is in the desired direction. If not, consideration could be given to 
increasing the time guidelines, publicizing them more widely, or strengthening them to 
incorporate real incentives. In addition, if data were collected in the state on a district-by- 
district basis, state level personnel could work with those districts which devote the least 
time to elementary science, helping to locate appropriate instructional materials, upgrading 
the skills of Uementary teachers, and perhaps having districts with stronger programs serve 
as models or sources of technical assistance. 

While the total amount of time spent on science and mathematics instruction at the 
junior high school level is no less than that devoted to other subject areas, the wty that 
instructional time is spent may be cause for concern. One of the most discouraging findings 
of the 1985-86 National Survey of Science and Mathematics Education was the substantial 
decline in the frequency of hands-on instruction in science. Fur example, the 1977 survey 
found that 72 percent of junior high school science lessons included lecture and 59 percent 
included hands-on activities, a difference of 13 percent. In 1985-86, 83 percent of the 
junior high school science lessons included lecture and only 43 percent included hands-on 
activities, a difference of 40 percent. 
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Mathematics educators may also be concerned about the pattern of instruction in 
many mathematics classes. Most mathematics lessons at the junior high school level 
include lecture (89 percent), discussion (90 percent), and seatwork assigned from the 
textbook (76 percent); only 20 percent include the use of manipulative materials, and only 
6 percent involve computers. Depending on their views of the educational process, if 
ndividual states find comparable results, they may wish to modify curriculum guidelines 
and/or recommend instructional materials that will encourage a different mix of 
instructional activities. 

Nationally, 47 of the 54 jurisdictions (50 states, District of Columbia, and 3 
territories) had course credit graduation requirements in science and mathematics as of 
June, 198/; six left graduation requirements up to the local districts, and one specified 
requirements only at the 7-9 level. According to CCSSO data, there has been a significant 
increase in science and mathematics requirements in recent years. From 1980 to 1984, 36 
states increased graduation requirements in math and 33 states increased requirements in 
science. From 1984 to 1987, seven more states increased graduation requirements in math 
and seven also increased requirements in science. Additionally, in that period of time 
seven states implemented an advanced or honors diploma specifying higher levels of 
science and math credits [Biank and Espenshade, 1987b]. By tracking course enrollment 
data in their states, pohcymakers will be able to determine the types of courses that 
students are taking to meet these increased requirements (e.g., if the enrollments in 
advanced courses are increasing or if local districts are creating general and vocational 
alternative courses) and to make policy modifications if needed. 
2. Educational Equity 

The 1985-86 National Survey of Science and Mathematics Education found 
considerable differences in science and mathematics course offerings among rural, urban, 
and suburban schools, and among schools of varying sizes. For example, only 22 percent of 
small high schools offer a course in calculus, compared to 57 percent of large high schools. 
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Policy alternatives available to states that have these types of distributional inequities 
might include instruction .ia telecommunications or forging cooperative arrangements 
among neighboring school districts to pool their enrollments. Similarly, states that find 
large percentage "-male and minority students opting 0 ut of higher level science and 
mathematics courses might wish to implement one or more of the inteivention programs, 
e.g., MESA or SECME, that have proven effective in encouraging the participation of these 
groups. 

3. Pre-Service Teacher Preparation 

The CCSSO inventory found considerable variation among the states in teacher 
certification requirements in science and mathematics. For example, while 21 states and 
jurisdictions require from 18 to 24 semester course credits in mathematics for secondary 
certification, 14 require from 27 to 34 credits, and four require from 36 to 45 credits. Some 
states require twice as many courses as othe 3 require. There is even greater variation 
among science certification requirements: 42 states offer a bi --field certification, with 
requirements ranging from 18 to 60 credit hours of science. In addition, in many of these 
states teachers can receive endorsements to teach specific science courses such as biology, 
earth science, chemistry, physics, or general science; again, course credit requirements for 
thtse endorsements vary considerably from state to state. 

Also, there is a discrepancy between the current preparation of many science 
teachers and their teaching assignments. Many science teacher education programs are 
designed to prepare teachers for a particular discipline, e.g., biology, chemistry, or physics. 
However, Aldridge [1987] reports that 20 percent of biology teachers, 35 per -ent of 
chemistry teachers, and 83 percent of physics teachers teach only one or two sections in 
that particular discipline. Most schools are not large enough to offer full teaching loads of 
chemistry and physics. Even in biology, the most popular high school science course, only 
40 percent of the schools offer 5 or more ections. The reality, therefore, is that most 
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science teachers will be assigned to teach courses in at Last two, and often three or more, 
disciplines. 

The 1985-86 National Survey of Science and Mathematics Education found that *he 
problem of teacher misalignment in science is in fact mainly due to science teachers 
teaching science courses outside their area of primary specialty; relatively little of the "out- 
of-field" teaching is due to English and Social Studies teachers teaching science. States 
may wish to rethink their teacher certification requirements in light of these data. For 
example, it may make considerably more sense to prepare "chemistry/physics" teachers 
with an adequate grounding in both of these fields than to have teachers who are well 
versed in only one of these areas routinely teaching both. 

Data about the age of science and mathematics education teachers can be 
particularly helpful as "early warning" signals to states. While nationally the average age of 
science and mathematics teachers is about 40, with about 17 percent age 30 or younger and 
15 percent age 50 or older, some states report very different age distribution patterns. By 
tracking changes in these distributions, and noting the differences among race/ethnic 
groups and subject areas, states will be able to determine when special recruiting efforts 
such as scholarships and loan-forgiveness programs may be beneficial and where they 
should be targeted. 
4. In-Service Education 

In two national surveys, one in 1977 and the other in 1985-86, elementary teachers 
were astad to rate their qualifications for teaching mathematics, science, social studies, and 
reading. In 1977, most elementary teachers indicated they felt very well qualified to teach 
reading (63 percent); corresponding figures were 49 percent for mathematics, 39 percent 
for social studies, but only 22 percent for science. By 1985-86, the differences in teacher 
perceptions about science and other subjects were even more marked. While 82 percent of 
the teachers indicated they feel very well qualified to teach reading, 67 percent to teach 
mathematics, and 47 percent to teach social studies, only 27 percent of elementary teachers 
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feeJ very well qualified to teach life sciences, 15 percent physical sciences, and 15 percent 
earth/space sciences. 

While modifications in pre-service preparation may improve the situation for future 
teachers, states must still somehow deal with the fact that large numbers of elementary 
teachers are not very comfortable teaching science. In the 1985-86 survey, half of all 
elementary teachers reported having received no. in-service education in science or the 
teaching of science in the last 12 months. Similarly, the 1985-86 survey found large 
numbers of science and mathematics teachers who feel unprepared to use computers as an 
instructional tool; percentages ranged from 50 percent of secondary mathematics teachers 
to 67 percent of elementary science teachers. 

The CCSSO inventory found that most states have staff development and in-service 
programs designed specifically to improve the knowledge and skills of science and 
mathematics teachers; many of these activities have been supported by federal funds under 
Title H of the Education for Economic Security Act. One of the three priority indicators of 
the CCSSO Science/Mathematics Indicator Project will provide data about the extent to 
which science and mathematics classes are being taught by teachers who are not certified in 
these fields, a problem that can be addressed in part by in-service education efforts. The 
Department of Education's Schools and Staffing Survey plans to provide additional 
information on in-service education programs, including the rates of teacher participation 
by sex, race/ethnicity, and type of community within each state [Center for Education 
Statistics, 1987]. State policymakers will be able to use these types of herniation along 
with other data about needs in science and mathematics education to help decide where to 
target in-srrvice resources. 

STATE-BY-STATE INDICATORS: THE NEXT STEPS 

While e'io .o develop a system of state-by-state indicators are still in their 
infancy, it is encouraging to note the emphasis on the collection of policy-relevant data. 
Each of the three priority indicators of the CCSSO Science/Mathematics Indicators Project 
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will provide information that can be used to hel« make decisions at the state level. The 
success of the "indicators movement" will be judged in the future by the extent to which 
these data are used to identify and implement changes needed to improve science and 
mathematics education. 
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